Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Reply

In Brodeur, Cook, and Heyes (2020) we present evidence that instrumental variable (and to a lesser extent difference-in-difference) articles are more p-hacked than randomized controlled trial regression discontinuity design articles. We also find no (i) published in the top five journals different; (ii) “revise resubmit” process mitigates problem; (iii) things improving through time. Kranz Pütz (2022) apply novel adjustment address rounding errors. They successfully replicate our results with exception of shakiest finding: after adjusting for errors, bunching test statistics difference-in-difference is now smaller around 5 percent level coincidentally larger at 10 level). (JEL A14, C12, C52)